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Abstract 

This article explores the relationship between open educational resources (OER) created by students for 
use by other students, the long-term sustainability of the movement toward OER, and the success of 
students who use OER created by other students as part of their core curricular materials. We begin by 
providing definitions and a broader context for thinking about the possibility of student-created OER. We 
then describe a course context in which student-created OER have been slowly integrated into an online 
class over several years and examine the impact on student learning associated with their introduction. 
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Definitions and Context 

As defined by the William and Flora Hewlett Foundation, open educational resources or OER are: 

teaching, learning, and research resources that reside in the public domain or have been released 
under an intellectual property license that permits their free use and re-purposing by others. Open 
educational resources include full courses, course materials, modules, textbooks, streaming videos, 
tests, software, and any other tools, materials, or techniques used to support access to knowledge. 
(Hewlett, 2016, para. 7) 

While a simple library or Google Scholar search demonstrates that OER have been the topic of hundreds of 
research articles over the past decade, recent research suggests that most postsecondary faculty are 
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unaware of the existence of OER. For example, survey work by Allen and Seaman (2016) shows that only 
25.6% of US faculty are either “Aware” or “Very Aware” of OER. Consequently, most US faculty have yet to 
be introduced to the idea of open educational resources. 

When faculty initially hear about OER, their first impulse is often to ask questions like how do they get 
created? Even with the involvement of major philanthropy, how can the creation of OER be sustained over 
the long term? The topic of development models that can sustainability support the creation of OER has 
been discussed at length for over 10 years. OECD commissioned a series of articles on the topic, including 
papers by Downes (2006), Dholakia, King, and Baraniuk (2006) and Wiley (2006). These articles list and 
describe several potential revenue models for the ongoing acquisition of funding to support the creation 
and upkeep of open educational resources. Many of the projects described in the OECD papers were 
constantly searching for funding to continue operations, and not all of them succeeded in that task (Parry, 
2009). 

As they maintain a largely financial focus, the commissioned papers overlook the possible role of students 
in the creation and maintenance of OER. In describing their primary user groups, Dholakia et al. (2006) 
list authors, instructors, and “students, who consume the educational materials, and learn” (p. 8). This 
quote is representative of a broader view of students as novices without the expertise necessary to 
meaningfully contribute to the creation and maintenance of OER. However, when framed in this manner, 
the issue of the potential role of students as creators of OER closely resembles the history of the Nupedia 
and Wikipedia projects. 

Novices as Producers of Excellent Content 

Jimmy Wales and Larry Sanger launched Nupedia in 2000. The goal of the Nupedia project was to publish 
free, openly licensed encyclopedia articles that were written and peer reviewed by experts. This proved to 
be a slow and laborious effort, and fewer than 30 articles were produced in the project’s first year. Wales 
and Sanger then created Wikipedia in 2001 as a sandbox in which anyone could contribute to the creation 
of articles that would be subjected to Nupedia’s peer review process before publication. As Sanger (2005) 
later wrote: 

My initial idea was that the wiki would be set up as part of Nupedia; it was to be a way for the public 
to develop a stream of content that could be fed into the Nupedia process. I think I got some of the 
basic pages written—how wikis work, what our general plan was, and so forth—over the next few 
days. I wrote a general proposal for the Nupedia community, and the Nupedia wiki went live 
January 10. The first encyclopedia articles for what was to become Wikipedia were written then. It 
turned out, however, that a clear majority of the Nupedia Advisory Board wanted to have nothing 
to do with a wiki. Again, their commitment was to rigor and reliability, a concern I shared with 
them and continue to have. Still, perhaps some of those people are kicking themselves now. They 
(some of them) evidently thought that a wiki could not resemble an encyclopedia at all, that it would 
be too informal and unstructured, as the original WikiWikiWeb was (and is), to be associated with 
Nupedia. They of course were perfectly reasonable to doubt that it would turn into the fantastic 
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source of content that it did. Who could reasonably guess that it would work? But it did work, and 
now the world knows better, (para. 33) 

The success of Wikipedia as a “fantastic source of content” was completely unexpected, even by its creators. 
Having witnessed an existence proof of the creation of excellent novice-produced content in the Wikipedia 
context, there is reason to believe it may be possible that novices (students) might be able to produce 
excellent open content in educational settings as well. 

Shirky (2010) argues that the success of projects like Wikipedia can be attributed (in part) to a combination 
of what he calls a “cognitive surplus” and the manner in which technology allows us to put this previously 
unproductive free time to better use. Where the potential of a quiet evening at home was previously limited 
to playing cards or listening to the radio, that same evening can now be spent contributing to Wikipedia or 
participating in a broad range of citizen science projects online. While students may not have a large 
quantity of free time at their disposal to volunteer to participate in projects that make the world a better 
place, they do spend an incredible amount of time doing homework. We pause here to provide the reader 
with a sense of how much time students spend doing homework. 

According to the US Department of Education’s Digest of Statistics (Department of Education, 2016), there 
were over 20 million students enrolled in a degree-granting postsecondary institution in the US during the 
fall of 2014. Let us assume that each of these students enrolled in one three credit course that term. For 
each of the three hours those students spend in class each week they can expect to do two hours of 
homework outside of class, for a total of six hours of homework per week over a typical 15-week course. 
Multiplying 15 total weeks times six hours of homework per week times 20 million students equals 
1,800,000,000 hours spent collectively on homework per semester. Because many students take more than 
a single course each semester, this estimate of approximately 2 billion hours per semester spent doing 
homework is conservative but adequate for our purposes. 

Renewable Assignments: For the Students, By the Students 

Following Wiley (2013), we define disposable assignments as those assignments that both faculty and 
students understand will ultimately be thrown away. Essays and practice problems are examples of 
assignments that frequently fit into this category - students spend significant time and energy doing this 
work, faculty spend significant time and energy grading this work, and then it gets returned to students who 
throw it away. While there is likely a learning benefit to students who complete these disposable 
assignments, literally throwing away 2 billion hours of work each semester seems like a missed opportunity. 

To contrast disposable assignments, we define renewable assignments as those which both provide a 
learning benefit to the student and result in OER that provide a lasting benefit to the broader community. 
Assigning students to create, revise, or remix OER results in an artifact that can be used both to evaluate 
student learning and increase the diversity of voices and perspectives available for study by later students. 
While not all disposable assignments can be converted easily into renewable assignments, if even a tiny 
fraction of the 2 billion hours (or more) of work done on disposable assignments could be put to more 
productive use, this 
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Murray (2008) provides an excellent example of a renewable assignment. Instead of writing essays that 
would be graded by their instructor and then thrown away, students enrolled in SPAN312: Murder, 
Madness, and Mayhem: Latin American Literature in Translation at the University of British Columbia 
did their writing on Wikipedia where others could read and benefit from their work: 

This was not a course about Wikipedia but rather, as with my other courses, its focus would 
continue to be on Latin America and on the reading of a set number of Latin American literary 
texts.... These books are neither short nor simple; most of the students' time would be spent reading 
these hefty tomes, and most of the class time spent explaining and discussing them. So there was 
little direct discussion of Wikipedia in the classroom. Rather, the assignment was that, in groups, 
the students should edit (and in a couple of cases create) Wikipedia articles on the texts and authors 
that we were covering, and that over the course of the semester they should bring these articles up 
to what in Wikipedia parlance is called "featured article" status, (para. 5-6) 

Of the 12 articles written by students, three achieved Featured Article status and eight achieved Good Article 
status. 

Randall, Johnson, West, and Wiley (2013) provide another example of a renewable assignment in their 
discussion of an openly licensed textbook on the topic of project management written for use in a business 
school context. A faculty member teaching project management in a graduate program on educational 
technology adopted the textbook understanding that much of the context and examples provided in the 
book would be inappropriate for his students. Over several semesters, he assigned students to work with 
him to collaboratively rewrite the book, adding examples relevant to educational technology, integrating 
new video case studies they produced, and making other changes that further improved the book for 
educational technology students. 

Azzam et al. (2017) describe a renewable assignment in which fourth year medical students improve the 
quality of health-related content in Wikipedia. They describe 43 students engaging in research on specific 
medical topics and making 1528 edits to 43 Wikipedia articles about high interest health topics like diabetes, 
dementia, and stroke. Collectively, these articles have been viewed over 22 million times since they were 
improved by the students. Assam et al. demonstrate how students can be assigned work which both 
demonstrates their mastery and provides lasting benefit to the community. 

OER and Quality 

In addition to questions about sustainability, faculty often wony about the quality of OER. According to 
Allen and Seaman (2016), 28% of faculty list “Not high-quality” as one of the three most important barriers 
to their use of OER in courses they teach (though “quality” is never defined). The concern is greater (32%) 
among faculty who describe themselves as being aware of OER than among those who are not aware (19%). 
This is somewhat surprising given the empirical work on the impacts of OER adoption on student success. 
For sake of clarity, for the remainder of this article we limit the meaning of the “quality” of educational 
resources to their effectiveness in supporting learning. 
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Hilton (2016) reviews nine empirical studies published in peer reviewed journals that focus on the 
effectiveness of OER assigned in place of commercial textbooks as the primary resource for courses. After 
acknowledging that assigning OER in place of commercial materials saves students a significant amount of 
money, he writes: 

In total 46,149 students have participated in studies relating to the influence of OER on learning 
outcomes. Only one of the nine studies on OER efficacy showed that the use of OER was connected 
with lower learning outcomes in more instances than it was with positive outcomes, and even this 
study showed that the majority of the classes were non-significant differences. Three had results 
that significantly favored OER, three showed no significant difference and two did not discuss the 
statistical significance of their results. In synthesizing these nine OER efficacy studies, an emerging 
finding is that utilizing OER does not appear to decrease student learning, (para. 55) 

While remaining silent on the issue of whether OER use is associated with better learning outcomes, Hilton 
concludes that OER use is not associated with decreases in student learning. This is an incredibly important 
finding as it means that faculty who adopt OER in place of traditional commercial materials can expect to 
save their students a significant amount of money without harming student learning. 

But what about student-created OER? It appears from the above examples that it is possible for students to 
participate in the sustainable creation and maintenance of OER, but none investigate the impact on student 
learning when students use OER created by other students. Hilton demonstrates that using OER in place 
of commercial materials does not harm student learning, but none of the OER used in the studies he reviews 
were created by students. Consequently, we do not know what happens when these two strands are brought 
together. How effective are student-created OER in supporting learning by later students? If student- 
created OER are unable to effectively support learning by other students, they are of little value even if they 
can be sustainably created and maintained. 

While the examples of renewable assignments above are powerful, they describe the work of upper-level 
undergraduate and graduate students who may not be accurately described as novices. To better 
understand the question of the learning impacts of using OER produced by other students we identified a 
setting in which the student creators would be more likely to be true novices - OER created by high school 
students for other high school students. 


An Example of Students Learning from Student-Created OER 

Institutional Context: Mountain Heights Academy 

Mountain Heights Academy opened in 2009 under the name Open High School of Utah as a virtual charter 
school with a mission to serve students in 9-12 grades from across the state of Utah (Open High School of 
Utah, 2009). In 2013, the school added 7th and 8th grades and changed its name to Mountain Heights 
Academy. The school currently serves 550 full-time students and 200 part-time students in grades 7-12. 
Each student receives a laptop from the school and access to online courses hosted in the Moodle LMS. 
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Full-time teachers are available both asynchronously and synchronously for a total of four hours per day 
during their office hours to work with students individually or in small groups (Mountain Heights Academy, 
2014). The entire curriculum is built using open educational resources, and the resulting courses are 
publicly released for universal access worldwide. Students are invited to contribute their own OER to the 
content of their courses if they choose to do so. Mountain Heights Academy thus provides a rich setting in 
which to study the impact of student-created OER on student learning. 

Course Context: Digital Photography 

The Digital Photography course was created and released for the 2010-2011 school year and has been taught 
each subsequent year. This course is a basic introduction for students in the field of digital photography. 
The course includes units on camera functions, lenses, image editing and output, photography genres and 
art history, and composition. After completing the course, students should be able to use artistic images to 
communicate ideas and themes make educated decisions in purchasing photography equipment with the 
functionality they desire. 

The course is a semester course organized into two quarters. In the State of Utah, students are able to take 
the course for either Fine Arts or Career Technology Education credit. At the beginning of each term 
(quarter), there is a section titled “Navigating the Course” that contains tutorials and guides for students, 
highlighting key skills instructors have identified as necessary for student success in the class. Each quarter 
of the course is structured into nine weekly modules. Lesson and assessment pieces for each module are 
due on the Friday of each assigned week. Students are given multiple summative and formative assessment 
opportunities in the course to demonstrate mastery of concepts, including portfolio creation, essays, 
multiple choice quizzes, self-evaluation, journaling, and teacher observation. The weekly modules cover at 
least one major concept and for each concept, there are specific learning objectives and linked assessments. 
On average, a weekly module contains seven pages, one assignment, and one quiz. 

Student-Created OER in Digital Photography 

Three opportunities are available for students to add OER to the course each year. First, students are 
encouraged with extra credit to release their own photos with a Creative Commons license. This involves 
creating their own account on a photo sharing site like Flickr, uploading their photos, and applying the 
Creative Commons license. These photos are then evaluated and the strongest photos of a particular concept 
are selected to be added to the course as examples. Many students have taken advantage of this opportunity 
since the course was first offered in 2011-2012. Second, at certain points in the course where students have 
struggled in the past, all students are given the opportunity to raise their grade if they create a tutorial video 
for a particular assignment. These tutorial videos are evaluated and a few of these are selected to be placed 
into the course. Approximately five students each year take advantage of this additional opportunity. 

Finally, after demonstrating high levels of masteiy, strong students are offered the opportunity to be a 
teaching assistant for the upcoming semester. These students create notes for each unit, study guides for 
exams, tutorial videos, and review presentations and games that are all added to the course and released as 
OER. For all materials created by teaching assistants the instructor thoroughly vets the material, edits for 
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formatting and quality, and then implements the OER in the course. As future teaching assistants work with 
the instructor, they revise and remix material that was created by prior teaching assistants. 

Learning Impacts of Student-Created OER 

When Digital Photography was first taught in 2011-2012 it contained no student-created OER. During the 
2012-2013 school year the first collection of student-created OER was added, and additional student- 
created OER were added again in 2013-2014 and 2014-2015. By 2014-2015, between 5% and 10% of all 
course content was comprised of student-created OER. 

During the 2011-2015 period the instructor, assignments, and grading criteria all remained stable. The only 
significant changes made to the course were the additions of student-created OER. This setting provides a 
compelling opportunity to ask questions and draw preliminary conclusions about the impact of student- 
created OER. 

We examined student performance data from a cross section of class assignments including understanding 
basic photography concepts, applying photography processes to students’ own compositions, synthesizing 
prior knowledge to create new work via tutorials and study of art history, and analyzing and evaluating 
students’ own and others’ work to give constructive feedback. We then ran a t-test for each assignment to 
determine if the average grade for the assignment had changed significantly over the period. The average 
grades and for each assignment are presented in Table 1. 

Table 1 


Average Assignment Grades and T Test Values 



2011-2012 

(n = 103) 

2014-2015 

(n = 78) 

t 

P 

P < 0.05 

Filters 

0.591 

0.766 

-2.617 

0.010 

* 

Popping 

0.517 

0.686 

-2.387 

0.018 

* 

Lenses 

0-554 

0.706 

-2.511 

0.013 

* 

Shutter Speed and ISO 

0.592 

0.731 

-2.166 

0.032 

* 

Pixlr 

0.587 

0.718 

-1.850 

0.066 


Cropping 

0.728 

0.831 

-1.695 

0.091 


Light and Exposure 

0.587 

0.586 

0.004 

0-997 


Camera Modes 

0.715 

0.714 

0.015 

0.988 


Black and White 

0.746 

0-733 

0.209 

0.834 


Rules of Composition 

0.824 

0.708 

1-756 

0.081 


All Assignments 

0.638 

0.719 

-3-829 

0.000 

* 


As shown in Table 1, average grades rose a statistically significant amount for 4 of the 10 assignments. The 
average grade for all assignments increased by eight points from 2011-2012 to 2014-2015. We calculated a 
t-test to determine if the difference between the average course grades in 2011-2012 and 2014-2015 were 
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statistically different from each other. The results indicate that the 2014-2015 scores are statistically 
significantly higher than the earlier scores (t = -3.829, p < 0.001). 


Discussion, Limitations, and Implications 

The Digital Photography course was neither designed nor taught with the goals of providing a context for 
research on the sustainability of student-created OER or the impact of learning from student-created OER. 
Consequently, all of the conclusions described below must be considered preliminary and in need of 
additional research. 

Students in the Digital Photography course were invited to create OER as an optional, extra credit activity. 
Very few students took advantage of this opportunity. This suggests that offers of extra credit alone may not 
be sufficient to incentivize a sustainable level of OER creation and maintenance. If the opportunity to create 
OER were integrated into required assignments as an option for students, as we saw in the Murray (2008), 
Randall et al. (2013), and Azzam et al. (2017) examples, we suspect a greater amount of OER would be 
created. The degree to which students will create and contribute OER in extra credit settings versus required 
assignment setting merits additional study. 

While the overall trend of student grades increasing in a statistically significantly manner as additional 
student-created OER were added to the course and other course elements remained unchanged is 
encouraging, additional research is needed before broad claims regarding the degree to which student- 
created OER supports better student learning can be made. Given that student-created OER made up only 
five to ten percent of the course content by the end of the fourth year, our result is particularly surprising. 

Future studies addressing similar questions may benefit by framing their research as dosage studies that 
look for relationships between the amount of student-created OER present in the course and degrees of 
difference in student performance across dosages. It seems unlikely that student performance will continue 
to increase as more student-created OER are added to the course. For example, while we found that when 
5% to 10% of course content was comprised of OER, students experienced an 8-point improvement in their 
overall assignment score, we do not necessarily believe that if 15% to 20% of the course had been comprised 
of student-created OER we would have seen a 16-point improvement in student grades. The relationship 
between the amount of student-created OER added to a course and the resulting changes in student 
performance needs much more investigation. 


67 



A Preliminary Exploration of the Relationships Between Student-Created OER, Sustainability, and Students Success 

Wiley, Webb, Weston, and Tonks 

References 

Allen, E., & Seaman, J. (2016). Opening the textbook: Educational resources in U.S. higher education, 
2015-16. Babson Survey Research Group. Retrieved from 

http://www.onlinelearningsurvev.com/reports/openingthetextbook2016.pdf 

Azzam, A., Bresler, D., Leon, A., Maggio, L., Whitaker, E., Heilman, J. ...McCue, J. D. (2017). Why medical 
schools should embrace Wikipedia: Final-year medical student contributions to Wikipedia 
articles for academic credit at one school. Academic Medicine, 92(2). Pp. 194-200. DOI: 

10.1097/ACM.0000000000001381 

Department of Education. (2016). Table 304.10: Total fall enrollment in degree-granting postsecondary 
institutions, by state or jurisdiction: Selected years, 1970 through 2014. Retrieved from 

http://nces.ed.gov/programs/digcst/dis/tablcs/dtih 204 .io.asp?current=ves 

Dholakia, U., King, J., & Baraniuk, R. (2006). What makes an open education program sustainable? The 
case of connextion. Organisation for Economic Cooperation and Development. Retrieved from 

https://www.oecd.org/edu/ceri/ 2678 i 78 i.pdf 

Downes, S. (2006). Models for sustainable open educational resources. Organisation for Economic 

Cooperation and Development. Retrieved from https://www.0ecd.0rg/edu/ceri/.267816Q8.pdf 

Hewlett. (2016). Open educational resources [Web blog]. Retrieved from 

http://www.hewlett.org/programs/education/open-educational-resources 

Hilton, J., IIL (2016). Open educational resources and college textbook choices: A review of research on 
efficacy and perceptions. Educational Technology Research and Development, 64(4). pp 573- 
590. DOI: 10 . 1007 /sil 422 - 0 l 6 -Q 424 -Q 

Open High School of Utah. (2009). Mountain heights academy: Charter application. Retrieved from 

http://www.mountainheightsacademv.org/wp-content/uploads/2014/08/2012.02-Charter- 

Application-MHA.pdf 

Mountain Heights Academy. (2014). Mountain Heights Academy: Annual report - 2013-2014. West 
Jordan: UT: 2014. Retrieved from http://www.mountainheightsacademv.org/wp- 
content/uploads/2Qif;/oi/Annual-Report- 20i2-20i4-FINAL.pdf 

Murray, J. B. (2008). Was introducing Wikipedia to the classroom an act of madness leading only to 
mayhem if not murder? [Web blog]. Retrieved 1 May, 2017 from 

https://en.wikipedia.Org/wiki/User:Jbmurrav/Madness 

Parry, M. (2009, September 3). Utah State U.’s OpenCourseWare closes because of budget woes [Web 
blog]. The Chronicle of Higher Education. Retrieved from 

http://www.chronicle.com/blogs/wiredcampus/utah-state-us-opencourseware-closes-because- 

of-budget-woes/ 7 Qi .2 


68 















A Preliminary Exploration of the Relationships Between Student-Created OER, Sustainability, and Students Success 

Wiley, Webb, Weston, and Tonks 

Randall, D. L., Johnson, J. C., West, R.E., & Wiley, D. (2013). Teaching, doing and sharing project 
management: The development of an instructional design project management textbook. 
Educational Technology, 53(6), 24-28. 

Sanger, L. (2005). The early history of Nupedia and Wikipedia: A memoir [Web blog]. Retrieved from 
https://features.slashdot.org/storv/oF;/o 4 /i 8 /i 642 i 3 /the-earlv-historv-of-nupedia-and- 

wikipedia-a-memoir 

Shirky, C. (2010). Cognitive surplus: How technology makes consumers into collaborators. New York, 
NY. Penguin. 

Wiley, D. (2006). On the sustainability of open educational resource initiatives in higher education. 
Organization for Economic Cooperation and Development. Retrieved from 

http://www.oecd.org/dataoecd/. 2 .' : t/Q/. i: t 864 c ; 447 .pdf 

Wiley, D. (2013). What is open pedagogy? [Web blog]. Retrieved from 

https://opencontent.org/blog/archives/ 2 Q 7 f; 


D Athabasca 
University 



69 







